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(57) Abstract 

The present invention discloses a novel system for controlling the inbound and outbound data packet flow in a computer network. By 
controlling the packet flow in a computer network, private networks can be secured from outside attacks in addition to controlling the flow 
of packets from within the private network to the outside world. A user generates a rule base which is then converted into a set of filter 
language instruction. Each rule in the rule base includes a source, destination, service, whether to accept or reject the packet and whether to 
log the event. The set of filter language instructions are installed and executed on inspection engines which are placed on computers acting 
as firewalls. The firewalls are positioned in the computer network such diat all traffic to and from the network to be protected is forced to 
pass through the firewall. Thus, packets are filtered as they flow into and out of the network in accordance with the rules comprising the 
rule base. The inspection engine acts as a virtual packet filtering machine which determines on a packet by packet basis whether to reject 
or accept a packet. If a packet is rejected, it is dropped. If it is accepted, the packet may then be modified. Modification may include 
encryption, decryption, signature generation, signature verification or address translation. All modifications are performed in accordance 
with the contents of the rule base. The present invention provides additional security to a computer network by encrypting communications 
between two firewalls between a client and a firewall. This permits the use of insecure public networks in constructing a WAN that includes 
both private and public networic segments, thus forming a virtual private networic. 
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A SYSTEM FOR SECURING THE FLOW OF AND SELECTIVELY 
MODIFYING PACKETS IN A COMPUTER NETWORK 


BACKGROUND OF THE INVENTION 

This application relates, in general, to a method for controlling computer 
network security. More specifically jt relates to an easily alterable or expandable 
method for computer network security which controls information flow on the 
network from/to external and internal destinations. 

Connectivity and security are two conflicting objectives in the computing 
environment of most organizations. The typical modem computing system is built 
around network communications, supplying transparent access to a multitude of 
sen/ices. The global availability of these services is perhaps the single most 
important feature of modem computing solutions. Demand for connectivity comes 
both firom within organizations and from outside them. 

Protecting network services from unauthorized usage is of paramount 
importance to any organization. UNIX woricstations, for example, once connected 
to the Internet, will offer all the services^which it offers another station on the next 
table to the entire world. Using current technology, an organization must give up 
much of its connectivity in order to prevent vulnerability, even to the extent of 
eHminating all connections to the outside worid or other sites. 

As the need for increased security grows, the means of controlling access 
to network resources has become an administrative priority. In order to save cost 
and maintain productivity, access control must be simple to configure and 
transparent to users and applications. The minimization of setup costs and down 
time are also important factors. 

Packet filtering is a method which allows connectivity yet provides security 
by controlling the traffic being passed, thus preventing illegal communication 
attempts, both within single networks and between connected networks. 

Current implementation of packet filtering allows specification of access list 
tables according to a fixed fomiat. This method is limited in its flexibility to express 
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1 Lned in that particular table. This method does not allo« the 
:r:::ro:llt ptotL. or se,v.= are no. spewed . the or..ha, 

Another method of ..P— Packet f^m is taiiorir,, «« oornp-r 
n^ratiho system code manual^ in every strategic point in the or9an.zat,on. Th« 
^2 is TI^ by ns flexibii^ to future chan,es in networK topology new 
:rrc!l rid services and to future security threats, it requires a iarge 
tf 1 .V experts mod«ying prcpr^tary computer proo-ar.^. n.aK,n. . 

— r rnrrseltnis^nce com^unica.. 
rr :reh neL. were em.oyed . P... — 

=::rhrprrj=^ 

::;rilmet as a mediator. Once connected to a .oca, Intern, provrder, 

,re*»o,i<s can quickly connect to any destination around the «odd. 
pnvate netwoms can quicmy la called a virtual private 

A private networi. that uses some public segments .s called a vi P 

^ r A VPN is significantly less expensWe and rr^m fle»ble than a 

network (VPN). A VPN , g ^ 
dedicated private network. Each of the pnvara i„e«)ensive 
,o a local internet provider. Adding new connections ,s srmpie and »,e>cpens.«^ 
to a local intemei p „, , vPN is that H is Insecure because of its 

However, a major disadvantage of a VPN is that 

. Th. Internet connection exposes the enteipnse la 
insecure segments. The Internet co 

following two dangers: (1) unauthonzed Internet 
networi<s (break-ins) and (2) eavesdropping on and tampenng w* 
communication as they pass. hrough^e^^rnet ^^^^ 

The security risks involved in communicating ove 
deterred enterprises from taking ful, advantage of VPNs. Doing business over 
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Internet (e.g., transfening funds, obtaining and verifying credit information, selling 
and delivering products) requires a reliable and effective security solution. 
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♦;«n c*.eks to provide an improved flexible. 
Accordingly, the present .nventon seeKs to pr ^ ^ 

*u^^ **#hirh contro s information iiaw o 
easlly-alterable seeunty method g. p^^nt Application 

„et«ork to that desaibed in oopendmg coass,gned 

08/168,041 . infftrmation flow/ on the network 

a<""«^- , . is to control information flow by means 

Yet another obieC o. the mvenf on ,s to »n ^^^^ ^ 

o, a pacKe, filter capable of examining even, pacKet 

n«,e in the system, ^ paoKe, ^^^^^'^^^^ ,y tt,e pacKet 

---"'''^^:ir:::n;>: f passm^ the pacK. on. « .t is 

fflter wherein the packet filter «.p ^^^.^^ 
preauthorized, prefer^ly after a " n--*^;^^ ^ p3,,et filter module 

obiec. Of — 3 given seou^ty policy at a 
:r torHal: r::X. pa.. .here.n the packet . passed 
only if it passage is preauthonzed. ^^^^^^ a 

^""Tltlsi:: : re::^^ administrator «nho. the 

computer ne*«ork wh,ch « ea* ^ 

"^^ror:::--ristop™v.eanimpr^ 

Check. . . provide the ability to modify the 

vet another oh)ect ; ::;ra.dress, accep«ng eternal 

packet by any of encryp.ng 0^ rnod-^ 9 communication. 

inputs as criteria for accephng, rejecting ^^^^^^ 
Another object of the present mvemion " 

as the Internet, 

for securing the flow of data over insecure pubhc networ . 
) thusfomiingaVPN. .^^^^^^^ ^^^^^ -.3 provided a 

According to an aspect of the p encrypting them, inter- 

to secure transactions over networks oy eno yn 
computer system to secure u«» 
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connect various networks with different addressing schemes and provides ways to 
pass packets of information only when the source of the communication is 
authorized and detecting the validity of traffic through the network while minimizing 
the information required to achieve it, preferably in a fail-safe architecture. 
5 There is provided in accordance with a preferred embodiment of the 

present invention a method of inspecting and selectively modifying inbound and 
outbound data packets in a computer network, the inspection and selective 
modification of the data packets occunring in accordance with a security rule, the 
method including the steps of generating a definition of each aspect of the 

10 computer network inspected by the security rule, generating the security rule in 
terms of the aspect definitions, the security rule controlling at least one of the 
aspects, converting the security rule into a set of packet filter language 
instructions for controlling an operation of a packet filtering module which inspects 
and selectively modifies the data packets In accordance with the security rule. 

15 coupling the packet filter module to the computer network for inspecting and 
selectively modifying the data packets in accordance with the security rule, the 
packet filter module implementing a virtual packet filtering machine, and the 
packet filter module executing the packet filter language instructions for operating 
the virtual packet filtering machine to either accept or reject the passage of the 

20 data packets into and out of the network computer and selectively modify the data 
packets so accepted. 

Further, the aspects can include network objects, network services or 
network services. In addition, the object definitions include the address of the 
object and the filter language instructions of the step of converting are in the form 

25 of script and further comprise a compiler to compile the script into the instructions 
executed in the step of executing. 

Still further, both the steps of generating the aspects of the network and of 
the security rule are defined graphically and the selective modification is chosen 
from the group consisting of encryption, decryption, signature generation and 

30 signature verification. 

There is also provided in accordance with a preferred embodiment of the 
present invention, in a security system for inspecting and selectively modifying 
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inbound and ou*ound data pacKate in a computer ne*«orK, the ^ 
Zeo^, and se,ec«ve, n,od»yin. the data packets .n the ^-puter m 
LcLance with a securfty rule, .Here each aspect o, the compu^r e^o* 
inspected hy the secuhty rule has been p^vlously deflned, the secunty ru e be 
rLus, defined In terms o, the aspects and converted into pacKet f«e 
anguage instruotU^^s. a method tor operating the security system — the 
Lps I providing a pacKet fliter module coupled to the computer ne^-ori. ir. « 
.east one en«y o, the computer netvyor. to be inspected by the secur,ty rule me 
paoKet f,«er mod^e implemenUng a virtual pacKet Altering rr^chine inspect, ng and 
Llecuve^ modifying the data pacKets passing into and out of the computer 
nittrK, ar,d the pacKet filter module executing the pacKet ««er language 
Ict^ns for operating the .rtual pacKet «,te.ng machine to e«her a^P. o 

, .1,. rf=.t=. nadcets Into and out of the computer networK ana 
reject the passage of the data pacKeis iniu « 

to selectively modify the data pacKets so accepted. 

Also provided in accordance with a preferred embodiment of me present 
invention, in a security system for inspecUng and selectively modHying .nbound 
and outbound data pacKets in a computer net«orK, the sec^Hty s^tem ,n« 
and selectively modifying the data pacKets in the computer ne.wod< ,n^ n^ 
^ a security rule, where each aspect of the computer net«orK .nspected by the 
2n has been previously defined, the security rule being previously 
die! in .em,s of the aspects and converted into pacKet filter ^nguage 
nllns, a method for opera«ng the security system deluding the steps of 
rig a pacKet filter module coupled to the computer networK in at .ast one 
^nt^y Of the computer net„orK to be controlted by the security rule, the pacKet 
« lodule emulating a virtual pacKe. fiKering machine inspecting and se^^vely 
m^ing the data pacKets passing IMo and out of the computer net«orK the 
Ta^^t filr module reading and executing the pacKet filter language '.struCons 
orZ pacKet filtering operafions, storing the results obUined in the step o, 
: Z and Jecu^ng the pacKet filter language instrucfions in a stooge d^ce 
and the pacKet filter module u«l^ng the stored results, from previous .nspecbon. 
or c^erLg the pacKet fi«er module to accept or reject the passage of the data 
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packets into and out of the computer network and to selectively modify the data 
packets so accepted. 

In addition, there is also provided in accordance with a preferred 
embodiment of the present invention, in a security system for inspecting and 
selectively modifying inbound and outbound data packets in a computer network, 
the security system inspecting and selectively modifying the data packets passing 
through the computer network in accordance with a security rule, where each 
aspect of the computer network controlled by the security rule has been previously 
defined, the security rule being previously defined in terms of the aspects and 
converted into packet filter language instructions, the security system including a 
packet filter module coupled to the computer network, the packet filter module 
operating in accordance with the security rule, the packet filter module 
implementing a virtual packet filtering machine inspecting and selectively 
modifying the data packets passing into and out of the computer network, and 
processing means for reading and executing the packet filter language instruction 
integral with the packet fitter module, the processing means operating the packet 
filtering module to either accept or reject the passage of the packets into and out 
of the computer network and to selectively modify the data packets so accepted. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is an example of a network topology; 

Fig. 2 shows a security system of the present invention applied to the 

network topology of Figure 1 ; 
5 Fig. 3 shows the computer screen of the network administrator of Figure 2 

in greater detail; 

Fig. 4 is a flow diagram of the subsystem for converting graphical 

information to filter script; 

Fig. 5 is a flow diagram of an information flow on a computer network 

10 employing the present invention; 

Fig. 6 is a flow diagram of the operation of the packet filter shown in Figure 

5; ^ . 

Fig. 7 is a flow diagram showing the virtual machine operations shown in 

Figure 6; 

15 Fig. 8 is a flow diagram of the data extraction method of Figure 7; 

Fig. 9 is a flow diagram of the logical operation method of Figure 7; 

Fig. 10 is a flow diagram of the comparison operation method of Figure 7; 

Fig. 11 is a flow diagram of the method of entering a literal value to 

memory; 

20 Fig. 12 is a flow diagram of a conditional branch operation; 

Fig. 1 3 is a flow diagram of an arithmetic and bitwise operation; 
Fig. 14 is a flow diagram of a lookup operation; 
Fig. 1 5 is a flow diagram of a record operation; 

Fig. 16 is a high level block diagram illustrating an example configuration 
25 employing firewalls constructed in accordance with the present invention; 

Fig. 17 is a high level block diagram illustrating the data transferred 
between two firewalls during a session key exchange; 

Fig. 18 is a high level logic flow diagram illustrating the process perfomned 
by a firewall in transmitting a packet using encryption to another firewall during a 
30 session data exchange; 
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Fig. 19 is a high level logic flow diagram illustrating the process performed 
by a firewall in receiving an encrypted packet from another firewall during a 

session data exchange; 

Fig. 20 is a high level block diagram illustrating the data transferred 

between two firewalls during a basic key exchange; 

Fig. 21 is a high level block diagram illustrating an example configuration 
employing a client personal computer and a firewall constructed in accordance 

with the present invention; 

Fig. 22 is a high level block diagram illustrating the data transferred 
between a client personal computer and a firewall during a session key exchange; 
. and 

Fig. 23 is a high level block diagram illustrating the data transferred 
between a client personal computer and a firewall during a basic key exchange. 
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DETAILED DESCRIPTION 

securing Inbound and Outbound DaU Packet Flow 

Refeoing now to Figure 1, an example network topology ,s '"o""^" * = 
example tt,e main s«e 100 contains a system administrator func«on embod ed ,n 
Tr^ttion 102. This wod<sta«on Is coupled to the network which ..u es 
: irons 104. router 1 10 and gateway lOa. Router 1 10 is coupled via sa^ e 
112 to a remote site via gateway 122. Gateway 106 is coupled via router 108 to 

le lime. The remote s«e 120 comprises workstations 124 which are coup ed 
the internet. The re ^ conf,gurat,on 

to the network and VB gateway l^ lo 

Shown herein is chosen as an example only and is not meant to hm,t the ^ o 
• eLork on which the present ,nven«on can work. The number oo^;-;*- 
networks can take are virtually limitless and technicues for setung up these 
clllgurations are well known to those skilled in the art. The present ,nvent,on can 
Doerate on any of these possible configurations. 

Figure Shows the network of Figure 1 in which the present inven.on has 
' .een insllled. In Figure 2, elements also shown in Figure 1 have me same 

ce numerals. As shown, the system administrator 102 Includes a contro^ 
reference num medium 

module 210, a packet «.er geneiator ^- ^^^'^ L system administrator, 
212 Packet filters 204 have been installed on the system 
. wo kstations 104 and gateway 106. Gateway 106 has -"^^^'-^^^^ 

.nnec«on to the network and one on . — n to *e - ~ 
108 and 110 each have a programming scnpt table wnicn IS g 

. u- no oart of the present invention, and vmII not be 

security system, but which fonns no part of th p 

described in detail. These tables correspond to the ^blesjh 
,5 utir^ed to program routers, as is we,, known to those skilled in «.e art^^ 

packet filters 204 are also installed on the gateway 122 of the i^mote s 
120 one packet fiKer « installed on the connection between the -te,^'-^'^ 
le gateway 122, a second packet Alter instalted on the connection be^veen the 
IlTand gateway 122 and a third packet «.er is Instailed on the connect.n 
30 between the gateway and the network. 
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Information flows on the network in the fonn of packets, as is well known to 
those skilled in the art. The location of the packet filters in Figure 2 is chosen so 
that data flow to or from a particular object of the network, such as a workstation, 
i-outer or gateway can be controlled. Thus, workstations 104 each have a packet 
filter so that the information flow to/from these workstations is separately 
controlled. At the remote site 120, however, the packet filter is placed on the 
connection between the gateway 122 and the network, thus there is no individual 
control over the data flow to/from the workstations 124. If such individualized 
control were required, packet filters coiild be placed on each of the workstations 
124. as well. Each of the packet filters is installed at the time that the network is 
set up or the security system is installed, although additional packet filters can be 
installed at a later date. The packet filters are installed on the host device such as 
the workstation or gateway at which protection is desired. 

Each of the packet filters operates on a set of instructions which has been 
generated by the packet filter generator 208 in the system administrator 102. 
These instructions enable complex operations to be performed on the packet, 
rather than merely checking the content of the packet against a table containing 
the parameters for acceptance or rejection of the packet. Thus, each packet filter 
can handle changes in security rules with great flexibility as well as handle 
multiple security rules without changing the structure of the packet filter itself 

The system administrator enters the security rules via a graphical user 
Interface (GUI) which is displayed upon the monitor 206 and explained in more 
detail with respect to Figure 3. This Information is processed by the packet filter 
generator 208 and the resulting code is transmitted to the appropriate packet filter 
or filters in the network to perform the function that is desired. Control module 210 
enables the system administrator to keep track of the operations of the network 
and storage 212 can be utilized to keep logs of operations on the network and 
attempts of illegal entry into the network. The system operator can thereby be 
provided with full reports as to the operation of the network and the success or 
failure of the security rules. This enables the security administrator to make those 
changes that are appropriate in order to maintain the security of the network 
without limiting its connectivity. 

11 
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Figure 3 shows the computer screen 206 In Figure 2 in more detail. The 
screen Is broken into four windows, two smaller windows at the left side and two 
terger windows at the right side. Network objects and sen,lces are two aspects of 
the network Which must be defrned in the security method of the present invention. 
Window 304 is used to define network objects such as the workstations, gateways 
and other computer connected to the system. 1. is also possible to group 

various devices together such as, for example, the linance department, the 
research and development department, the directors of the company. 1. ,s *us 
possible to control data flow not only to indivWual computers on the network, bu 
also to groups of computers on the network by the appropriate placement of 
packet Alters. This allows the system operator have a great deal of flexU>i,^ .n *e 
managing of communications on the network, it is possible for example to have 
me chief financial offlcer as well as other higher ranking offidals of the company 
such as the CEO and the directors abte to communfcate direct^ «;^;'"^"^ 
group but filter out communications from other groups. It is also possible to allow 
electronic mail from all groups bu. to limK other requests for infomiation to a 
speoHied set o, computers. This a«ows the system operator to provide internal as 
well as externa, security for the network. The object defin«ion wouW .dude *e 
address o, the object on the network, as well as a name or group whether the 
Object is internal or external to «,e network, whether or no. a packet filter has been 
installed on mis object and a graphical symbol. The graphical symbol ,s used ,n 
connection with the rule base manager 302, 

Simllariy, nelwori< services are defined In block 306 on the screen. These 
network services can include login, route, syslog and telnet, for example, ach 
service is defined by generic and specific properties. The 9--= 
include the code string «,at IdenHfies «,e service, for example 'dpor. (destinabon 
port) Which IS equal to 23 for telnet. The code string that identifies the ,ncom,ng 
and outgoing packets are identified. Spec«c properties Include the name oU^ 
service, the port used to provide the service, the fimeout in seconds of how long a 
connectionless session may stay InacUve, that is, having no padcet transm^ed « 
eKher direction before assuming that the session is completed. Other elemer^ts ^ 
a service definWon might include the program number for RFC services and the 

12 


BNSDOC1D- <WO 9700A71A2J>> 


wo 97/00471 PCT/IL96/00017 

outbound connections for accepted services that use connectionless protocols 
such UDP. The graphic symbol and its color are specified. 

Block 302 is the rule base manager which allows the new security rule to 
be entered into the system in a graphical manner, thus freeing the system 
5 administrator from having to write code to implement a particular security rule or to 
change a security rule. Only four elements are required to enter the new security 
rule into the system. The first element is the source of the data packet and the 
third element is the destination of the packet. The second element is the type of 
service that is involved and the fourth element is the action that should be taken. 

10 The action that can be taken includes accept the packet in which case the packet 
is passed from the source to the destination or reject the packet In which case the 
source is not passed from the source to the destination. If the packet is rejected, 
no action can be taken or a negative acknowledgment can be sent indicating that 
the packet was not passed to the destination. In addition, a further element which 

15 can be specified is the installation location for the rule which specifies on which 
objects the rule will be enforced (see Figure 2). If an installation location is not 
specified, the system places the packet filter module on the communication 
destination by default. These objects are not necessarily the destination. For 
example, a communication from the Internet and destined for a local host must 

20 necessarily pass through a gateway. Therefore, it is possible to enforce the rule 
on the gateway, even though the gateway is neither the source nor the 
destination. By entering the data with acronyms or graphic symbols, each rule can 
quickly be entered and verified without the need for writing, compiling and 
checking new code for this purpose. Thus, the system administrator need not be 

25 an expert in programming a computer for security purposes. As long as the 
service is one of the services already entered into the system, the computer 
serving as the host for the system administrator function will process the 
information into a set of Instructions for the appropriate packet filter, as described 

in greater detail below. 
30 Block 308 is a system snapshot which summarizes the setup and 

operations of the security system. It is not required to practice the present 
invention. The system snapshot displays a summary of the system using graphical 
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.y„,bo>s. The sun,™^ can InCude, for e^omple, «,e hos. icon, host narne, m|e 
Le nan,e. which ,s .he nan,e o. tt,e f„a containing the ruie t>ase an. *e date 
the rute base was »,sta,ted on the host, .t can a.» show the sutus o, the ost 
Zcm whether or not there have i=een oommunica^ons with the host as „e„ as 
, the numl«r of paclcets inspected by, dropped and logged by the host 

F gure 4 shows a flow chart o, the subsystem for corwerSng the ,n.onn3.on 
on the GU, to a f,i.er script which contains the ruies utii^ed ,or the pacKet «ter n 
Te pLerred embodiment, the output o, the f,«er schpt generator is comprled ^ 
ZL code Which is then impiemepted by the pac^. fi-ter module, as described 

'° '"" The subsystem 400 starts a. 402, proceeds to biocK 404 which ^ obtains 

X ^^ II The first rule is the first line on the screen in which a 

the first ru e from the GUI. The nrsi ruic i=. 

:i secuhty ruie has been ^enU.^, as shown in Figure 3. Oon^ Pr»e^= 
to biocK 406 in which code is generated to match «,e rule source --'--^^^^ 
,5 That is the source of the packet is entered Into the source code block as 
" Z-«ng one of obieC o, the system from which the data p^a - 
elnate. Con.ro, then passes to biock 403 in which co^e ^^"^^ ^^ 
desUnation code biod< to indicate which object of the natvork me data padce. s 
d^ne^ for. Control than passes to blook 4« in which code generate. 
.0 mth the rule services that weie chosen. The rule services have been de n d 
20 maicn inc lu^ defined will be defined at the 

previously and are stored within the system or. if not defined, wiu 

. . ^o..i=.tinn the service is entered Into the system. Control 
r :a:sro:ir4 Cr re . generated to accept or reject the packe. 
: e a o ks 40a, 403 and 410 were matched, that is, ^ .suits of the 

" ""^ „^ ,„ aceeo, or reject is based upon the action chosen 

,5 Checks were true. The achon ^ a»ept^ < ^ 

. «,e security rule. J^^^J"^ ^.^^ system, if no 

determines whether or not more rules are to o „ Mock 

more rules are to be entered into the system, «,e subsystem '^'^ 
413 ,f more rules are to be entered into .ha system, control passes to W * 41 3 
30 wl obums the next rule and passes control back to block ^O^;-*"^;™ 

process repeats and the next security rule, found on the next Una the GUI ,s 


processed. 

14 


BNSOOCID <WO 


970047 1A2„»_> 


^ PCTAL96/00017 
WO 97/00471 

Communication protocols are layered, which is also referred as a protocol 
stack. The ISO (International Standardization Organization) has defined a general 
model which provides a framework for design of communication protocol layers. 
This model serves as a basic reference for understanding the functionality of 
existing communication protocols. 

ISO MODEL 


Layer 

Functionaiity 

Example 

7 

Application 

Telnet. NFS. Novell NCP 

6 

Presentation 

XDR ' 

5 

Session 

RPC 

4 

Transport 

TCP, Novel SPX 

3 

Network 

IP, Novell IPX 

2 

Data Link (Hardware Interface) 


1 

Physical (Hardware Connection) 



Different communication protocols employ different levels of the ISO model. 
A protocol in a certain layer may not be aware to protocols employed at other 
layers. This is an important factor when making security actions. For example, an 
10 application (Level 7) may not be able to Identify the source computer for a 
communication attempt (Levels 2-3). and therefore, may not be able to provide 
sufficient security. 

Figure 5 shows how a filter packet module of the present invention is 
utilized within, the ISO model. The communication layers of the ISO model are 

15 shown at 502 at the left hand portion of Figure 5. Level 1, block 504, is the 
hardware connection of the network which may be the wire used to connect the 
various objects of the network. The second level, block 506 in Figure 5 is the 
network interface hardware which is located in each computer on the network. The 
packet filter module of the present invention intercedes between this level and 

20 level 3 which is the network software. Briefly, for the sake of completeness, the 
other levels of the ISO model are level 4. block 510 which relates to the delivery of 
data from one segment to the next, level 5. block 512. synchronizes the opening 
and closing of a "session" on the network. Level 6, block 514 relates to the 
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can... . .a. vartou. compute, o. ne«,o., ana ,eve, 7, MocK 

^ TXi::^^^^ o" - - :r :o rn 

u . i =,nri 2 and then is diverted to the packet filter 520, shown 
passes through layers 1 and 2 and tnen 

^ «f Piaure 5 The packet is received in block b/^. m u 

on the right hand portion of Figure 5. P ^-termination is made as 

S.. .a pacKet . — : ; a^^LcHes .e ru.e, >t 

may be logged on the system 

™.e to ent. the sy.e. ^^J^^^^ ^ ,3c.. based 

.ocK 53. .n WHICH H .He de.s,on . to pass tHe paCet. 

upon the re^u— o, the ^ ^ ^ ^^^^ ^ ^ 

«,e packet is then passed to ^ "o 

pscket. a negat«e acknowledgment (NACK) .s sent a. W 

-J ««tr«i nflsses to block 530 where the pacKei is qiuhk 
.een chosen, a^ s«ar,. . an applica«on generates a packet 

is, it is not passed to ,,3,^3 the .SO mode, at level 

Which is .0 be sent 3n .den«ca. process except 

3, block 508 ^'^^^'^l.^ , a, btocK 506 and not leve, 3, 

^ , the pacKet ,s to be pa^e^^ ^ ^^^^ ^^^^ 

„ock 508. on level 2. the pad<et . ^^^^^ 

packet examined to see U m ^^.^ 

matches any packet regardless of the source 

-empty rule" only has an action, which is to drop the packet If no 

. ! d this L win be retrieved and wW be effective to drop the packet 

..empty rul. co.d, 0. cou.e. be wH^n ^ ^ ^ 

Referring to Figure 6. 600 is a ^^^^ 

desoripuons shown ,n [ 3,^„ i„ ^ose figures are 

30 module" as the tern, ,s "'^'^^'^' ^.T.J'no,.^^ '<> operate. Figures 11-15 
the minima, capabilities for the packet trner 
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show addition features which may also be included in the packet filter module, but 
are not required in the minimal definition of the term. 

The packet filter module is embodied In a "virtual machine", which, for the 
purposes of this application, may be defined as an emulation of the machine 
5 shown in Figures 6-1 0 residing in the host computer, which is a computer on the 
network. 

The virtual machine starts at block 602 in which the packet is received, 
which corresponds to block 522 of Figure 5. Control passes to block 604 in which 
the filter operations are obtained from the instruction a memory (not shown). 

10 These filter operations are the filter operations that have been generated by the 
packet filter generator 208 shown in Figure 2. Control then passes to block 604 in 
which the filter operations are obtained and then to block 606 in which the memory 
618 is initialized. In block 608, the first virtual machine operation is obtained and 
performed in block 610. The virtual machine contains a memory mechanism such 

15 as a stack or register 618 which may be utilized to store intermediate values. The 
utilization of this stack or register is shown in greater detail in connection with the 
table shown below. Control then passes to decision block 614 in which it is 
determined whether or not the stop state has been reached. If the stop state has 
been reached, the decision will have been made to accept or reject the packet, 

20 which decision is implemented at block 616. If the packet has been passed, the 
packet will proceed as shown in Figure 6. If the packet is rejected, it will be 
dropped and a negative acknowledgment may be sent as shown in blocks 528 
and 530. If the stop state has not been reached in block 614, the next operation is 
obtained in block 616 and the process repeats starting with block 610. 

25 The type of operations that can be perfonned in step 5, block 610 are. 

shown more clearly in Figure 7. In Figure 7. block 610 and block 614 are identical 
to the blocks shown in Figure 6. Connection 613 is interrupted by three operations 
which are shown in parallel. For the operation that is to be performed in block 610, 
control will pass to the appropriate block 702, 704 or 706 in which that task will be 

30 perfomied. In block 702 data extraction will be perfomied. in block 704 logical 
operations will be performed and in block 706 a comparison operation will be 
performed. As shown at the right hand portion of Figure 7, other blocks can be 
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added in parallel to the operations capable of being performed by the virtual 
machine. The subset shown as blocks 702. 704 and 706 are the essential 
elements of the virtual machine of the present invention. These elements are 
shown in greater detail in Figures 8. 9 and 10. respectively. Additional elements 
5 which may optionally be included in the operations capable of being performed by 
the virtual machine are shown in Figures 1 1-15. respectively. 

The data extraction block 702 is shown In greater detail in Figure 8. The 
process starts at block 802 and control passes to block 804 in which data is 
extracted from a specific address within the packet 806. This address is taken 
10 from the stack memory 618 or from the instruction code. The amount of data 
extracted is also detemiined by the stack memory or the instruction code. The 
extracted data is put into the memory stack 810 at block 808. The process 
temiinates at block 812. In these figures, control flow is shown by arrows havmg a 
single line whereas data flow is shown by arrows having double lines. 
15 Figure 9 shows logical operation 704 in greater detail. The logical operation 

starts at block 902 and control passes to block 904 in which the first value .s 
obtained from the memory 906. In block 908 a second value is obtained from the 
memory and the logical operation is perfom^ed in block 910. If the logical 
operation is true, a one is placed in the memory 906 at block 912 and if the logical 
20 operation is false, a zero is placed in the memory 906 at block 914. The process 

terminates at block 916. 

The third and last required operation for the virtual machine is shown .n 
greater detail in Figure 10. The comparison operation, block 706. starts at block 
1002 and control passes to block 1004 in which the first value is obtained from 

25 memory 1006. Control passes to block 1008 in which a second value is obtained 
from memory 1006. A comparison operation between the first and second values 
takes place at block 1010. If the comparison operation is true, a one is placed m 
memory 1006 at block 1012 and if the comparison operation is false a zero .s 
placed in memory 1006 at block 1014. The process tem^lnates in block 1016. 

30 The following operations are not shown in Figure 7 but may be added at the 

right side of the figure at the broken lines and are connected in the same manner 
as blocks 702. 704 and 706. that is. in parallel. Figure 1 1 shows the entenng of a 
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literal value into the memory. The process starts at block 1 1 02 and control passes 
to block 1106 in which the literal value is obtained from the instruction code. The 
value is placed into the memory at block 1108 and the process ends at block 
1110. 

5 A conditional branch operation is shown in Figure 12. The process starts at 

block 1202 and control passes to block 1204 in which the branch condition, taken 
from the Instruction code, is checked. If the branch condition is true, the value is 
obtained from the memory stack 1206 at block 1208 and checked at block 1210. If 
the results of the comparison in block 1210 is true, the next step is set to N and 
10 the process terminates at block 1216. if the comparison in block 1210 is false, the 
process terminates at block 1216. If the branch condition is false, at block 1204, 
control passes directly to block 1214. 

* 

An arithmetic or bitwise operation is shown in Figure 13. The process starts 
at block 1302 and control passes to block 1304 in which the first value is obtained 

15 from memory 1306. The second value is obtained from memory 1306 at block 
1308 and an arithmetic or bitwise operation is performed on the two values 
obtained from the memory 1306 in block 1310. The result of the arithmetic or 
bitwise operation is placed in the memory in block 1312 and the process 
terminates in block 1314. 

20 Figure 14 illustrates a lookup operation which Is useful if data needs to 

passed from a first set of instructions implementing a security rule to a second set 
of instructions for a second security rule. As shown in block 606 of Figure 6, the 
memory is initialized whenever a new security rule is processed. Therefore, 
information placed in the memory by a first security rule will not be available for 

25 use by a second security rule. In order to overcome this problem, a separate 
memory 1410 is supplied which contains Tables 1-3 which can be utilized for this 
purpose. The entry of data into the tables is shown in Figure 15 and described 
below. The lookup operation starts at 1402 and control passes to 1404 in which 
values are obtained from memory 1406. Control passes to block 1408 in which 

30 data is obtained from Tables 1-3 at block 1410 by searching the values in the 
referred table. Control passes to block 1412 in which a decision is made as to 
whether the block is in the table. If the decision is yes, a one is placed in memory 
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1406 at block 1416. If the decision Is no. a zero Is placed In memory 1406 at block 
1414 The process terminates at block 1418. 

' Referring to Figure 15. the process starts at block 1502 and control passes 
to block 1504 in which values are obtained from memory 1506. Control then 
passes to block 1508 In which values obtained from memory 1506 are placed m 
the appropriate locations in Tables 1-3 at block 1510. Control passes to block 
1512 in which a decision is made as to whether or not the storage values in the 
table has succeeded. If the storage has succeeded a one is placed in memory 
1506 at block 1516. If the process has not succeeded, a zero is placed in memonr 
1506 at block 1514. The process terminates at block 1518. 

An example of a security rule is implemented using the packet filtenng 
method of the present invention will now be described utilizing as an example the 
security rule to disallow any Telnet services In the system. Telnet Is defined as 
being a TCP service and having a specific TCP destination port. It will be identified 
by having a TCP protocol value of 6 in byte location 9 of the packet and by having 
a destination Telnet protocol number of 23 In byte location 22 of the packet, the 
value being a two-byte value. This .s found in every Telnet request packet. 

The first operation in the table shown below is to extract the IP protocol 
from the packet location 9 and place this in memory. As shown in the "Memory 
values" column at the right side of the table, this value. 6. is placed at the top of 

the stack. . ^ . . . 

The second operation, the TCP protocol (port) number, which .s stated to 

be 6 above, is placed at the second location in memo^. In step 3. the values of 

the first two layers of the stack are compared, obtaining a positive result. 
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Drop Telnet Process 


# Packet 


Filter Code 


Virtual Machine Operation 


Memory Values 
(Stack Order) 


1 pushbyte 
[91 


2 push 6 


eq 


4 pushs [22] 


5 push 23 


eq 


and 


Extract Operation: Extract IP protocol 6 
number from packet location 9 to 
memory 

Enter Literal Value to Memory: Put TCP 6 
protocol number in memory 

Comparison Operation: Compare IP 1 
protocol to TCP, obtaining a positive 
result 

Extract Operation: Extract 1 

TCP protocol number from packet 

location 

22 to memory 

Enter Literal Value to Memory: Put 1 
TELNET protocol number in memory 

Comparison Operation: Compare TCP 1 
protocol to TELNET, obtaining a 
positive 
result 

Logical Operation: Check if protocol 1 
both 

TCP and TELNET are matched 


23 


23 23 


8 btrue drop Conditional Branch Operation: If 

memory value is true, branch to drop 

state 
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The values of 6 at the top two layers of the stack are deleted and a 1. 
indicative of the positive result, is placed at the top of the stack. In step 4. the TCP 
protocol number for packet location 23 is extracted and placed in the memory 
location at the second layer of the stack. In step 5. the literal value which is the 
Telnet protocol nurr^ber is placed into the memory at the third layer of the stack. In 
step 6. the memory iaye.^ 2 and 3 containing the TCP protocol for Telnet .s 
compared with the expected value, obtaining a positive result. The values of the 
second and third layers of the stack are deleted and replaced by a 1 . indicative of 
the positive result. In step 7. a logical operation is perfomied to see if both the 
TCP and Telnet have been matched. This is detemiined by a AND operation. In 
this case the result is positive and the ones in the first two layers of the stack are 
deleted and replaced by a 1 indicative of the positive result. In step 8. a 
conditional branch operation is perfom^ed in which if the memory value is tme. the 
program branches to the drop state. In this case, the result is tme and the 
program branches to the drop state in which the Telnet request is not passed. 
Thus the rule to drop Telnet has been Implemented. 

Encrypting Data Flow - An Introduction 

As stated earlier, long distance communications between enterpnses. 
branch offices and business partners have become an essential part of modem 
day business practice. Utilizing the present invention, virtual private networks 
(VPNs) can be constructed over insecure public networks such as the Internet to 
provkle secure and flexible communications. 

The modification of packets by. e.g.. encryption of outbound packets, 
decryption of Inbound packets, signing of packets or address translafon .s 
perfom^ed by the packet filter module. The decision whether to modify a packet .s 
detem^ined from the rule base. All modifications, i.e.. encryption, decryption, 
signing and address translation are perfom^ed on a selective basis in accordance 
w^ the contents of the rule base. For encryption, for example, to occur, a rule .n 
the rule base must explicitly call for encryption to occur on packets which have a 
particular source, destination and service type. The encryption instructions are 
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translated into the packet filter language that are installed and executed on the 
virtual packet filter machines in the network. 

4 

As described previously, the packet filter module detemiines whether a 
packet is rejected or accepted. If rejected, the packet is dropped. If accepted, the 
5 packet may be modified in a number of ways. Example of types of possible 
modifications include, but are not limited to, encryption, decryption and address 
translation. The following describes in detail the encryption and decryption of 
packets that is selectively performed by the packet filter module. 

Notation Used Throughout 

10 The following notation is used throughout this document: 


Symbol 

Description 

g 

common root used for all Diffie-Hellman keys 

P 

rrimmnn mndulus used for all Diffie-Heliman kevs 

Spvt 

source private key 

Spub . 

source public key 

Dpvt 

destination private key 

Dpub 

destination public key 

B 

basic key 

TB 

truncated basic key 

A 

auxiliary key 

R 

session key 

E 

session data encryption key 

1 

session data integrity key 

M 

data portion of a packet 

P 

unencrypted password 

ENCx(Y) 

encrypt Y using X as the key 

DCRx(Y) 

decrypt Y using X as the key 

SIG(Y) 

signature of Y 
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Definitions of T rms Used Throughout 

The following definitions are helpful in understanding the operation of the 

present invention. 


Term 

plaintext 
cleartext 


Definition 

text that is not encrypted 

another term for text that is not encrypted 


ciphertext encrypted text 


key 


encryption 


decryption 


certification 


digital 
signature 


network 
object 

gateway 


firewall or 
firewalled 
network 
object 


a piece of infomiation known only to the sender and the 
intended recipient 

converting the plaintext of a message into ciphertext In order to 
make the message unintelligible to those without the key 
converting ciphertext into plaintext using the same key used to 
encrypt the message 

a trusted third party, known as a Certificate Authority (CA). from 
which one can reliably obtain a public key. even over an 
insecure communication channel, generates a certificate for the 
public key which can be verified by the recipient 
infomiation generated from the contents of the message itself 
and used by the recipient to verify the data integrity of the 
message and/or its origin 

a piece of hardware that is connected to a network and which 
has some interaction with the nehwork 

a network object that is connected to at least two networks and 
passes information between them 

a network object, usually a gateway or an end host, that 
secures the flow of inbound and outbound data packets on a 
computer network and also selectively modifies data packets in 
accordance with a security rule base 
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A high level block diagram illustrating an example configuration employing 
firewalls constructed in accordance with the present invention is shown in Figure, 
16. The example network shown in this figure will be used to explain the 
encryption capabilities of the present invention. The network configuration shown 
5 is only for illustrative purposes only. Once skilled in the art can adapt the present 

■« 

invention to other network configurations as well. Both host1 and host2 are 
connected to their respective private LANs, in addition, firewalU 1604 is coupled 
to hosti through its LAN and firewall2 is coupled to host2 through its LAN. Both 
firewalls are coupled to a public network 1606 such as the Internet. It is also 

10 assumed that the public network is insecure and cannot be trusted. Certificate 
Authority 1 (CA1) 1602 functions as tine certificate authority for hosti and firewalll. 
CA2 1612 functions as the certificate authority for host2 and firewall2. In other 
embodiments, there may be only a single CA that sen/es both firewalls. In either 
embodiment, the functions of the CA remain the same. The only difference is 

15 which CA the firewall uses to obtain public keys. 

It is desired that the communications between hosti and host2 be secured. 
The communications from hosti is routed to the Internet (i.e.. the public network) 
via firewall 1 which acts as a firewalled network object. Similarly, communications 
from host2 is routed to the Internet via firewall2 which also acts as a firewalled 

20 network object. In communications to host2, firewalll intercepts and encrypts the 
packets It receives from hosti enroute to host2. Firewall2 receives the encrypted 
packets destined for host2 and decrypts those packets. In the opposite direction, 
firewall2 encrypts the packets from host2 destined for hosti. FirewalU receives 
the encrypted packets, decrypts them and passes them to hosti . The encryption 

25 and decryption operations performed by firewalll and firewall2 are transparent to 
hosti and host2. 

Assuming hosti initiates the session with host2. it sends an Internet 
Protocol (IP) packet to host2., Firewalll will intercept the packet and determine that 
communications between hosti and host2 are to be modified in some way. e.g., 
30 encryption, decryption, address translation, etc. The decision is made separately 
for each connection based on information from all the ISO layers and based on 
information retained from previous packets. This decision process is termed 
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stateful multi-layer inspecBon (SMLI). Each firewall maintains a rule base that 
instructs the firewall how to handle both Inbound and outbound con,mun,catK,ns 
between network objects, as described in detail eariier. After determining that 
communicauons between host1 and host2 are to be encrypted or digitally signed 
frewalM temporarily parks the packet and initiates a session key exchange, wh,ch 
is described in more detail below. Before encrypted communications or s,gn,ng 
can take place, both sides need to agree on a shared key. This key is called the 
session key, R, and is generated anew at the stari of every session. It is important 
,0 note that only the communications between «rewall1 and firewalE is encrypted 
because of the use of the insecure Internet or public netwo*. The 
communications between host1 and flrewalM and between host2 and firewall2 are 
not encrypted because it takes place over private LANs which can be assumed to 
be private and secure. 

Session Key Exchange - Firevrall/Firewall 

A high level block diagram illustrating the data transtetred between two 
firewalls during a session key exchange is shown in F^ure 17. The following 
scheme is only one example of Implementing enc^pSon with SMLI and « no 
meant to limit the scope of the present inventton to other encryption techniques. It 
would be Obvious to one skilled in the art to adapt other encryption technK,ues to 
the SMLI process to carry ou.«,e teachings of the present inven«oa For examp e^ 
m an alternative embodh,ent the SKIP standard is utll^ed. To ,n*ate the 
enc^ption of data, firewalll first sends a request packet to host2. The .equest 
packet is sen. to hos.2 and not flrewall2 because firewalll may not know the IP 
address of the firewall that is responsible for host2. FirewalE intercepts th,s 
request packet and returns a reply packet The request and reply packets altow 
both sides to agree on shared session key R «.at will be used for a, 
communications to be encrypted between host1 and hos.2. As stated previously, 
only the communications between firewalll and firewall2 are actually enc^ted. 
in general, the sess»n key R is generated by the non-inrtiator (,.e., firewall2 

^ X- *:^r, onH i« etent encrvpted to the initiator (».e., 
) 1608) also called the destination and is sent encrypie 

ii^M tho <:r.ijrce Thls two packet exchange must occur 
firewain 1604) also called the source, mis iwu ho 
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before encrypted communications can proceed. After the encrypted session is 
established, state information is maintained in both firewalls and the original 
packet that was parked is now passed encrypted through the firewalls. The same 
session key R is used by firewall2 to encrypt packets that are sent from host2 to 
5 hostl. 

The session key exchange will now be described in more detail. In order to 
agree on a common secret session key R, the present invention uses a 'static' 
Diffie-Hellman scheme. Each Diffie-Hellman key comprises a private part and 
public part. Each side has its own private and public parts. The private key for the 
10 source (i.e., firewalU) and destination (i.e., firewall2) is SpvT DpvT, 
respectively. The public parts for source and destination are then defined as 
follows: 

Spub = g^'"" (mod) p 
Dpub = g"""^ (mod) p 

15 Both source and destination must know each others public key for the session key 
exchange to work. If one side does not know the other's public key or the key it 
does have is determined to be out of date, than a basic key exchange is triggered 
which is explained in more detail below. Both sides use each other's public key to 
derive at the basic key B. The source perfomris the following: 

20 B = {g^^ (mod) pf (mod) p = g^^^^"^ (mod) p 

Similarly, the destination performs the following: 

B = {g^P^ (mod) pf^ (mod) p = g^"^''^ (mod) p 

Thus, both sides share the basic key B. For use in encrypting the session key R, a 
truncated version of the basic key B Is generated, called TB. 
25 In general, each firewall maintains a table of bindings between Diffie- 

Hellman keys and firewalied network objects. In addition, a firewall must have a 
binding between IP addresses and such an object. In the configuration shown in 
Figure 17, a database within firewalU must be configured so that it knows of 
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mewana's existence. FirewalU must also Know that host2's enc^-pting firewall is 
fl,ewall2. F,re«a>l1 can have a l..t o. potential flrewails that may sen,e as 
enorypting firewalls for r,rewall2. The bindings and the network object database for 
each firewall are managed in static fashion by a separate management umt. 

,n ouie, to encrypt communications between firewalls, a firewall must have 
. Know^ge of its own basic phvate Key and the baste public keys of each 
fi,Bwalled netwoH. object it needs to communicate ^th. The basK public keys 
belonging to external firewalled network objects such as a firewall betongmg to a 
business partner must also be known in order for encrypted sessions to occur. 
This s,3«c binding of basic keys to firewalled network objects may already be 
established in a database internal to the firewall or it can be obtained on the fly 
' using the basic key exchange described betow. 

Once a common shared secret basic key B is agreed upon by the t«o 
firewalls, it is used to encrypt the ac^al key used for the session, i.e.. the sessK.n 
key R. The same session key R is used by both source and destination to encrypt 
the data from host! to host2 and from host2 to hostl. 

The elements of the request from the source to the dest,nat,on ,s shown 
above the right arrow in Figure 17. The cipher method comprises one or more 
encryption methods for encrypting me session data that the source ,s able to 
pe.l (e.g., DES, FWZ1, RC4. RC5, IDEA, Tripple-PES, etc.,. The key methc^ 
oomprises one or more enc^«on methods for er«rypting the session ke R *at 
the source is able to perfbm, (e.g.. DES, FW21. RC4, RC5, IDEA. Tnpple-DES^ 
etc.). The md method (i.e.. message d^est method, or the message inte nd 
method comprises one or more methods or algorithms for perfomn,ng da« 
integrity that the source is able to perfom, (i.e., MD5, SHA, etc.). The data integ^ 
Ja^y entails calculafing a cryptographic hash of a par, of or al, of «,e n^a e ^ 
The suggested source publte key ID identifies, via an ID number, the bas^c 
public key that the source assumes the desfina«on wi„ use. Ukewi^, me 
ggested dest,„a«on basic public key ID idenUfies the basic public key th^ the 
souL assumes «,e dest.na«on will use. If there are more than one possible 
firewalled networi> object seeing host2, the source will include 
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firewalled network object actually serves host2. Each suggested basic public key 
corresponds to a different firewalled network object. 

The request also comprises a challenge key C which is a random bit field 
chosen by the source (i.e., firewalU) which is used to thwart man in the middle 
5 attacks against the session key exchange or the session data itself. 

The destination (i.e., firewall2) receives the request packet and based on its 
contents generates a reply packet to be sent back to the source. The elements of 
the reply packet are shown above the left arrow in Figure 17. The reply packet has 
a similar format as the request packet with the exception of the challenge key C 
10 field replaced by a field holding the encrypted session key R. Each of the cipher 
method, key method and md method now have only one element rather than a list 
of options as in the request. The elements listed are the elements chosen by the 
destination from the options listed in the request. Similarly, the chosen source 
basic public key ID and the chosen destination basic public key ID both comprise 
15 a single key ID representing the key ID chosen by the destination from the option 
list sent in the request. 

The session key R that is sent in the reply actually comprises two keys: a 
session data encryption key E and a session data integrity key I. Thus, the 
session key R is defined as 

20 R = E + I 

The session key is a random stream of bytes that is generated for both the cipher 
method (i.e., encryption method) and the md method or method digest method. Its 
length is the sum of the key lengths needed by the cipher method and the md 
method. Once generated, a signature of the session key is obtained using the 
25 chosen md method, e.g., MD5, and represented by SIG(R). The combination of 
session R and SIG(R) are then encrypted using a key formed by the combination 
of the truncated basic key TB and the challenge C. thus forming 

ENCrrc*c)(R + SIG(R)) 

which is what is sent in the reply to the source. 
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The signature or hash chiecksum computation provides authentication to 
the source that the packet it received is indeed fomied by an entity that l<nows the 
basic key B thus providing strong authentication for the reply packet In addition, 
since the source chose the challenge key C, there is no possibilrty of replay. 

Session Data Exchange 

A high level logic flow diagram illustrating the process perfomied by a 
firewall in transmitting a packet using encryption to another firewall during a 
session data exchange is shown in Figure 1 8. Although not shown in the Figures, 
an alternative embodiment utilizes the IPSEC standard for performing session 
data exchange. As mentioned earlier, once the source and the destination agree 
on a session key R, encrypted communication between both firewalls can 
proceed. Interception of and modifications to the packets occur between level 2 
and level 3 of the ISO model. Communications occurring both ways is to be 
encrypted and decrypted using the same session key R. The packets that are sent 
out closely resemble normal TCP/IP packets. The packets do not include any 
information indicating whether the packets are encrypted or not and if so which 
key to use. This information only exists in the state maintained by the two 
firewalls. The encryption is perfonned in place without changing the length of the 
packet which serves to increase the efficiency and bandwidth of encrypted traffic, 
m general, each transmitted packet is divided into two parts, a cleartext part which 
is not encrypted and a ciphertext part which is encrypted. The cleartext part 
includes the IP header and the TCP/UDP header. The rest of the packet meaning 
its data M is encrypted using a combination of the session key R and an auxiliary 
key A computed from its cleartext part The process will now be described in more 
detail. 

The first step performed by a firewall in transmitting a packet is to generate 
an auxiliary key A from the cleartext contents of the packet itself (step 1800). The 
portions used depend on the type of packet and protocol (e.g.. RPC. UDP. TCP. 
ICMP. etc.) and may include the following fields, for example. IP-ID (only a 
portion). RPC-XID. RPC-PROGNUM. TCP-SEQ. TCP-ACK. UDP-LEN. ICMP- 
TYPE. ICMP-CODE and IP-PROTO. Next, the auxiliary key A. session data 
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integrity key i and the data portion of the packet M are placed in a buffer (step 
1802). A signature is then generated on the contents of the buffer using the md 
method (step 1804) and expressed as 

SIG(A + I + M) 

5 The bits of the signature generated are then placed in the packet header 

(step 1806). Adding. the signature bits to the packet is important for ensuring data 
integrity. Since the length of the packet is not modified some portions of the 
packet must be overwritten with the signature bits. Note that the signature bits are 
stored In the packet before the packet Is encrypted. For TCP packets a 28 bit 
10 signature is stored as follows: 

• the 8 LSBits of the signature replace the 8 MSBits of the IP-ID 

• the next 18 bits are. added to the TCP-CSUM field using I's complement 
arithmetic 

• the next 4 bits are stored In the unused TCP-X2 nibble (this is optional) 

15 For UDP packets a 32 bit signature is stored as follows: 

• the first 16 bits of the signature are added to the UDP-CSUM field using 
I's complement arithmetic; If the original UDP-CSUM field is zero, the UDP- 
SPORT and UDP-DPORT fields are added to the UDP-CSUM also using 1's 
complement arithmetic 

20 • the next 16 bits are stored In the UDP-LEN field 

Once the signature bits are stored In the packet, the data portion of the 
packet M is encrypted (step 1808). and can be expressed by 

ENC{E*A)(M) 

The encryption Is performed using the cipher method with a combination of the 
25 session data encryption key E and the auxiliary key A. Finally, the packet is 
transmitted over the public network (step 1810). 

A high level logic flow diagram Illustrating the process performed by a 
firewall in receiving an encrypted packet from another firewall during a session 
data exchange is shown in Figure 19. First, in order to verify the signature, the 
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auxiliary key A must be generated from the contents of the packet (step 1900). 
Then the packet's data portion M is decr/pted using the cipher method and a 
combination of the session data encryption key E and the auxiliary key A (step 
1 902). which can be expressed as 

DCR(E*A)(ENC(E*A)(M)) 

Next the signature bits are extracted from the packet header (step 1904). A 
signature on the auxiliary key A. session data integrity key 1 and the packet data M 
is then generated using the md method (step 1 906). and expressed as 

SIG(A + I + M) 

Then the two signatures are compared with each other (step 1908). If they match 
the packet is passed after replacing any data in the packet that was oven^ritten 
with signature data (step 1910). If the signatures do not match the packet .s 
dropped (step 1912). 

Basic Key Exchange 

As explained previously, in order to encorpt communications between 
firewalled network objects, a firewall must have knowledge of own private bas.c 
key and the public basic keys of each firewall it needs to communicate with. The 
public basic keys belonging to external firewalls such as a firewall belong.ng to a 
business partner must also be known in order for encrypted sessions to occur. 
This static binding of basic keys to firewalls can already be established .n a 
database internal to the firewall or it can be obtained on the fly using the basic key 
exchange. In addition, the basic keys may be updated on an infrequent bas,s to 
improve security. The present invention provides for basic public keys to be 
obtained on the fiy if they are not already in a database within the firewall. In 
general, a basic public key must be obtained if the source does not have 
knowledge of the destination's basic public key or the destination detem^ines that 
the destination basic pubic key used by the source is out of date. 

m either case, the exchange of the basic public key is certified in order to 
be sure as to the authenticity of the Diffie-Hellman key being transmitted. 
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Certification of messages. In general, serve to thwart man in the middle attacks 

against the system. 

The process of exchanging the basic keys will now be described in more 
detail. A high level block diagram illustrating the data transferred between two 
5 firewalls during a basic key exchange is shown in Figure 20. Whenever any of the 
two sides recognizes that either it does not have a valid key for its peer or that it 
has an outdated key it requests the other side to send it a certified basic key. 

The basic key exchange can be triggered in two ways depending on which 
side discovers that the basic public key has to be updated or exchanged. 

10 Typically, it will be the side that discovers it does not have the other sides' basic 
key. For example, referring to Figure 16, a basic key exchange will be triggered If 
the initiating side i.e., firewalH. discovers that it does not have the basic public key 
forfirewall2. In another scenario, firewall2, upon receiving a request from firewalU, 
sees that it has an outdated version of the basic public Itey for firewalU (by 

15 comparing what is in its database to the suggested basic public key sent in the 
request). The latter scenario is the one depicted in Figure 20. 

The elements of the basic key request are shown above the left arrow in 
Figure 20. The basic request comprises the source basic public key ID. 
destination basic public key ID, cipher method, key method and md method. 

20 These elements are identical to those discussed above in the section entitled 
Session Key Exchange - Firewall/Firewall. When a basic key exchange must 
occur, the side that wants the other to send it a certified key update or key sync 
will add a CA public key ID field to the request This new field indicates which key 
requires updating and is the ID of the certificate authority key (e.g., RSA key) by 

25 which firewall2 wants to receive the reply from firewalU. Upon receiving this 
message, firewalU will send its basic public key Sp^t, to firewall2 after certifying it 
with the CA public key against a certification that was made by the CA. 
Certification is the process of generating a digital signature of the basic public key. 
For firewalU , CA1 1602 generates the CA public keys for verifying firewalM 's basic 

30 public keys (Figure 16). In order for firewall2 to verify the signature, it must obtain 
the CA public key from CA1 , the certificate authority for firewaltl . 
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The elements of the response by flrewalll are shown above the right an-ow 
in Figure 20. The elements comprise the CA public Key ID, the source basic public 
key and the IP address or addresses of the source. In addition, the signature 
of the source basic public key Is sent, which can be represented by 

SIG(S,a,) 

in a preferred embodiment, the signature is generated by first generating 
an intermediate signature from the basic public key to be sent using the md 
method of generating digital signatures. Then, this intemaediate signature ,s ,nput 
,0 the RSA decrypting function to generate the signature that is finally transmitted. 
The IP address of the source (i.e., firewain) is included In order to venfy the 
binding between the firewall, i.e., flrewalll , and a basic public key (S„.„). 

upon receipt of the certificate from flrewalll , f.rewall2 can verify it usmg the 
CA public key. If it verifles correctly, flrewall2 updates its database with the new 
basic public key of flrewalll. Now, the session key exchange can be completed 
and session data can then to be communicated. 

Note that the basic public keys are communicated between each flrewa^l 
and its CA over secum communication channels. If there is more than a single CA 
the public key o, one CA is sent In the clear to the other CA. This message « 
either signed using a previous value of the CA public key or the new y obtamed 
CA public key can be verified by some other manual means, such as facsimile or 

telephone. 

Session Key Exchange - Client/Firewall 

AS described eariier, there is a gnawing business need for external access 
to corporate networks. More and more employees are working physically outside 
the corporate LAN or WAN environment but need to connect to it. The present 
invention provides the capabi,^ of verif^ng external users of a system and 
providing encrypted communications between the external user or client and the 

host system. . , 

A high level block diagram illustrating an example configuration employing 

3 client personal computer and a firewall constructed in accordance wUh the 
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present invention is shown in Figure 21. A personal computer (PC) 2100, called 
the source for purposes of explanation, is used by the client or external user to 
login to the host 2104 shown coupled to a LAN. The PC is coupled to a public 
network 1606 and communicates with the host via firewall 2102, called the 
destination or server for purposes of explanation. All communications between the 
PC and the host' is routed through the firewall. The PC is suitably programmed to 
perform the functions needed to login to the host and carry out encrypted 
communication between itself and the firewall. Similar to the configuration shown 
in Figure 1 6. encrypted communications is only between the PC and the firewall in 
the configuration shown in Figure 21. To the host, the firewall Is transparent and 
thinks data is coming straight from the PC. 

The session data exchange processes for client to firewall encryption are 
similar to those of firewall to firewall encryption. The differences lie, however, in 
the session key exchange and the basic key exchange processes. With firewall to 
15 firewall session key exchange, each session received a different session key. A 
session is not only a connection between two particular network objects but may 
include different services between the same network object. In contrast, the client 
initiates a session with the host and all communications between the client and the 
host during that session is encrypted using the same key. no matter what activities 
20 or sen/ices the client requests. In addition, in firev/all to firewall communications, 
both sides have each other's certified public key. In client to firewall 
communication, this is true only for the client, while the server identifies the client 
using a name/password pair sent to it by the client. 

A high level block diagram illustrating the data transferred between a client 
25 personal computer and a firewall during a session key exchange is shown in 
Figure 22. The elements sent in the request by the client are shown above the 
right arrow. The elements include a name, cipher method, key method, md 
method, password method, source basic public key Sp„i,. suggested destination 
basic public key ID. challenge key C, encrypted password and a signature. The 
30 name is used to identify the user who is currently using the client. The cipher 
method, key method and md method are as described earlier. The password 
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™mod indicates wWch encryption method to use ia encrypUng me password. The 
encrypted password can be expressed as 

« 

ENC(rB + C)(P) 

The source basic public .ey is always sen. as the flrewan does no. .aintaih a 
„s, of users and «,e,r associated basic public Keys. The data that ,s sem ,s sim^a 
1 data sent by «rewai,1 to flrewa,^ (Figure 20, as described ,n the sec^n 
entitled Basic Key Exchange - Firewal^irewall. The des«na.ion basic pub„o key 
,D is as was described above in the secUon entitled Session Key Exchange - 

Firewall/Firewall. . . . 

The Signature ^.nctions to ensure to the destina«o„ , the receding s,de^ 
the message was not modi^ed. The s^nature is generated by ^K,ng me 
entire contents of the recuest or message, represented as T ,n Bgure 22. except 
Z the Signature field, and combing T w.h «,e unenc^pted password and «,e 
truncated basic public key TB, expressed as the following 

siGcr + P + TB) 

The signature is added to the request and the request then sen. to the flre^ll. 
After receipt of .he request me «rew., knows me .ien.. basic pub . 

S I. can now generate the basic key B and me tn^noated basro Key TB. I. .hen 

!::aecryp. .he password P. Once P is known, me firewall can verify »e signature 
: htZ est. The firewall nex. genera.es a random session key R and en^^ 

R and me signa.ure of R using me m.nca.ed basic Key TB and me challenge C 

sen. in me reques. from me client and given by 

ENC(iB.o)(R*SIG(R)) 
A signature is then generated of .he content of me request denoted by U in F«ure 
22 in combination wim the truncated basic public Key TB, as given by 


S1G(U + TB) 

36 


wo 97/00471 PCT/IL96/00017 

The firewall then generates a reply whose elements are shown above the left 
arrow in Figure 22. The reply comprises the destination basic public key ID. the 
cipher method, key method and md method, encrypted session key and the 
signature. 

5 Once the session key is known by both the client and the firewall, the 

communications session can proceed between the PC and the host via the firewall 
and the encrypted communication between the PC and the firewall is transparent 
to the host. In order to reduce the number of key exchanges, the session key R Is 
used for all encrypted connections passing through the same firewall. After a 

10 predetemiined time duration, e.g.. several minutes, the session key R is dropped. 


■ Basic Key Exchange - Client/Firewall 

In contrast to firewall to firewall communications, a certified key exchange is 
only necessary to update the client with the firewall's basic public key. A basic key 
exchange may be triggered in either of two ways. The first, if the client does not 
15 have the firewall's basic public key or. second. If the firewall detennines that the 
basic public key used by the client in the request is-outdated. 

The process is similar to the basic key exchange as explained previously in 
the section entitled Basic Key Exchange - Firewall/Firewall. However, there are 
differences as explained below. If the client realizes that it does not have the 
20 firewall's basic public key. it substitutes a CA public key ID field for the destination 
basic public key ID field in the request. This is shown above the top right arrow in 
Figure 23 which is a high level block diagram illustrating the data transferred 
between a client personal computer and a firewall during a basic key exchange. 
This key ID is the ID of the certificate authority key (e.g.. RSA key) by which the 
25 client wants to receive the reply from the firewall. 

When the firewall receives the request from the client, it determines from 
the request whether the client is requesting the firewall's basic public key or the 
key ID in the request does not correspond to the firewall's basic public key. The 
elements of firewall's reply is shown above the left arrow. The reply comprises the 
30 original suggested destination basic public key ID, CA public key ID. destination 
basic public key Dpub, IP address of the destination and a signature. The original 
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destination basic public key is taken as is from the request. The signature of the 
destination basic public key Is sent, which is represented by 

SIG(Dp„b) 

In a preferred embodiment, the signature is generated by first generating 
5 an intennediate signature from the basic public key to be sent using the md 
method of generating digital signatures. Then, this intemiediate signature .s input 
to the RSA decrypting function to generate the signature that is finally transmitted. 
The IP address of the destination (i.e.. the firewall) is Included in order to verify the 
binding between the firewall, and a basic public key (Dpub)- 
10 Upon receipt of the certificate from the firewall, the. client can verify it using 

• the CA public key. If it verifies correctly, the client updates its database with the 
new basic public key of the firewall. 

After receiving the firewall's reply, the client sends back a message to 
complete the authentication. The elements of the message are shown above the 
1 5 bottom right arrow in Figure 23. The message comprises the password encrypted 
and a signature. Once the reply is received, the client can generate the basic key 
B and the truncated basic key TB. The client then encrypts the password P. 
expressed as 

ENC(TB * C)(P) 

20 The signature is generated using the md method on the combination of the 
contents of the original request sent to the firewall as shown above the right arrow 
in Figure 22. represented as T. the cleartext password P and the truncated basic 
public key TB. as expressed as 

SIG(T + P + TB) 

25 The encrypted password and the signature are then sent to the firewall. The 
session key exchange then completes and session data communications can 
begin. 
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While the invention has been described with respect to a limited number of 
embodiments, it will be appreciated that many variations, modifications and other 
applications of the invention may be made. 
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What is claimed is: 


1 A method of inspecting and seteotively modifying inbound and outbound 
.ata paoKets in a computer ne*»orK, the inspec«on and seiecSve n»->«-«°"^ 
sa« dau packets occurring in accordance with a security ruie. me method 

comprising the Steps of: . ^^^t^n 

genemting a definiUon of each aspect of the computer networi. .nspected 

by said security rule; 
generating said security -uie in tem,s of said aspect definitions, sa,d 

security rale controlling at least one of said aspects; 
converting said security rale into a set of pacKet fliter language instract»ns 

for controlling an operation of a packet filtering module wh,ch 

inspects and selectively mcdffles said data packets in accordance 

witti said security rule; 

coupling said packet filter module to said computer ne.wori< fo, inspecting 
and selectively modifying said data packets in accordance wrth saKl 
security rale, said packet filler module Implementing a virtual packet 
filtering machine; and . . ^. 

said packet filter module executing sak. packet filter language .nstract,ons 
for operating said virtual packet filtering machine to either accept or 
reject the passage of said data packets into and out of said netv«.ri. 
computer and selectively modify said data packets so accepted. 

2. The method according to claim 1. wherein said aspects include networi< 
objects. 

The method according to claim 1, wherein said aspects include network 


3. 

services. 


4. 

services. 


The method according to claim 2, wherein said aspects include networi. 
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5. The method according to claim 4, wherein said object definitions include 
the address of said object. 

6. The method according to claim 1, wherein the filter language instructions of 
said step of converting are in the form of script and further .comprising a compiler 
to compile said script into said instructions executed in said step of executing. 

7. The method according to claim 1 , wherein in both said steps of generating 
said aspects of said network and of said security rule are defined graphically. 

8. The method according to claim 1, wherein said selective modification is 
chosen from the group consisting of encryption, decryption, signature generation 
and signature verification. 

9. In a security system for inspecting and selectively modifying inbound and 
outbound data packets in a computer network, said security system inspecting 
and selectively modifying said data packets in said computer network in 
accordance with a security rule, where each aspect of said computer network 
inspected by said security rule has been previously defined, said security rule 
being previously defined in terms of said aspects and converted into packet filter 

* 

language instructions, a method for operating said security system comprising the 
steps of; 

providing a packet filter module coupled to said computer network in at 
least one entity of said computer network to be inspected by said 
security rule, said packet filter module implementing a virtual packet 
filtering machine inspecting and selectively modifying said data 
packets passing into and out of said computer network; and 

said packet filter module executing said packet filter language instructions 
for operating said virtual packet filtering machine to either accept or 
reject the passage of said data packets into and out of said 
computer network and to selectively modify said data packets so 
accepted. 
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10. The method according to claim 9 where^ said aspects Include network 
objects. 

11. The method according to claim 9 wherein said aspects include network 

services. 

5 12. The method according to clah, 10 wherein said aspects indude network 
services. 

13. The method according to claim 12 where', said obiect definitions include 
the address of said object. 

14. The method according to claim 9 wherein said virtual machine perfomis a 
10 data extraction operation. 

15. Tne method according to claim 14 wherein said virtual machine perfom^ a 

logical operation, 

16. The method according to da»T, 15 wherein said virtual machine perfomis a 
comparison operation. 

1 , The method according to claim 9. wherein said selective modification is 
Chosen from the group consisting of encryption, decryption, signature genera.on 
and signature verification. 

,„ a security system tor inspecUng and selectively modifying inbound and 
outbound data packets in a computer network, said security svstemjnsp-'^ 
and selectively mod»ying said data packets m said computer network n 
accordance w«h a security ruSe. where each aspect of sal con^u«r r^twork 
inspected by said security rule has been previously def»,ed, sa,d secun^^ ^ 
bZ previously defined in tem,s o, said aspects and converted into packet « ^ 
language instructions, a method for opera«ns said secuHty system compnsmg the 

" "providing a packet mter module coupled to said computer network in a. 

It one enaty Of saU computer network to be controlled by said 


15 17. 


20 


18. 
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security rule, said packet filter module emulating a virtual packet 
filtering machine inspecting and selectively modifying said data 
packets passing into and out of said computer network; 


5 


said packet filter module reading and executing said packet filter language 
instructions for performing packet filtering operations; 


storing the results obtained in said step of reading and executing said 
packet filter language instructions in a storage device; and 


10 


said packet filter module utilizing said stored results, from previous 
inspections, for operating said packet filter module to accept or reject 
the passage of said data packets into and out of said computer 
network and to selectively modify said data packets so accepted. 


19. The method according to claim 18 wherein said aspects include network 
objects. 


20. The method according to claim 18 wherein said aspects include network 


21- The method according to claim 19 wherein said aspects include network 
services. 

22. The method according to claim 21 wherein said object definitions include 
the address of said object. 

20 23- The method according to claim 18, wherein said selective modification is 
chosen from the group consisting of encryption, decryption, signature generation 
and signature verification. 

24. In a security system for inspecting and selectively modifying Inbound and 
outbound data packets in a computer network, said security system inspecting 
25 and selectively modifying said data packets passing through said computer 
network in accordance with a security rule, where each aspect of said computer 
network controlled by said security rule has been previously defined, said security 
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services. 
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rule being previously defined in terms of said aspects and converted into packet 
filter language instructions, said security system comprising: 

a packet filter module coupled to said computer network, said packet filter 
module operating in accordance with said security rule, said packet 
5 filter module implementing a virtual packet filtering rhachine 

Inspecting and selectively modifying said data packets passing Into 
and out of said computer network; and 
processing means for reading and executing said packet filter language 
instruction integral with said packet filter module, said processing 
10 means operating said packet filtering module to either accept or 

reject the passage of said packets into and out of said computer 
network and to selectively modify said data packets so accepted. 

25. The method according to claim 24. wherein said selective modification is 
chosen from the group consisting of encryption, decryption, signature generation 

1 5 and signature verification. 

Zet2^ A method substantially as claimed hereinabove and substantially as 

illustrated in any of the drawings. 
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